Trade-Off between Task Accuracy, Task Completion Time and Naturalness for Direct Object Manipulation in Virtual Reality

Virtual reality devices are used for several application domains, such as medicine, entertainment, marketing and training. A handheld controller is the common interaction method for direct object manipulation in virtual reality environments. Using hands would be a straightforward way to directly manipulate objects in the virtual environment if hand-tracking technology were reliable enough. In recent comparison studies, hand-based systems compared unfavorably against the handheld controllers in task completion times and accuracy. In our controlled study, we com-pare these two interaction techniques with a new hybrid interaction technique which combines the controller tracking with hand gestures for a rigid object manipulation task. The results demonstrate that the hybrid interaction technique is the most preferred because it is intuitive, easy to use, fast, reliable and it provides haptic feedback resembling the real-world object grab. This suggests that there is a trade-off between naturalness, task accuracy and task completion time when using these direct manipulation interaction techniques, and participants prefer to use interaction techniques that provide a balance between these three factors.publishedVersionPeer reviewe

Comparison of a VR Stylus with a Controller, Hand Tracking, and a Mouse for Object Manipulation and Medical Marking Tasks in Virtual Reality

In medical surgery planning, virtual reality (VR) provides a working environment, where 3D images of the operation area can be utilized. VR allows 3D imaging data to be viewed in a more realistic 3D environment, reducing perceptual problems and increasing spatial understanding. In the present experiment, we compared a mouse, hand tracking, and a combination of a VR stylus and a grab-enabled VR controller as interaction methods in VR. The purpose was to investigate the suitability of the methods in VR for object manipulation and marking tasks in medical surgery planning. The tasks required interaction with 3D objects and high accuracy in the creation of landmarks. The combination of stylus and controller was the most preferred interaction method. According to subjective results, it was considered as the most appropriate because it allows the manipulation of objects in a way that is similar to the use of bare hands. In the objective results, the mouse interaction method was the most accurate.publishedVersionPeer reviewe

Deep Learning Method for Mandibular Canal Segmentation in Dental Cone Beam Computed Tomography Volumes

Accurate localisation of mandibular canals in lower jaws is important in dental implantology, in which the implant position and dimensions are currently determined manually from 3D CT images by medical experts to avoid damaging the mandibular nerve inside the canal. Here we present a deep learning system for automatic localisation of the mandibular canals by applying a fully convolutional neural network segmentation on clinically diverse dataset of 637 cone beam CT volumes, with mandibular canals being coarsely annotated by radiologists, and using a dataset of 15 volumes with accurate voxel-level mandibular canal annotations for model evaluation. We show that our deep learning model, trained on the coarsely annotated volumes, localises mandibular canals of the voxel-level annotated set, highly accurately with the mean curve distance and average symmetric surface distance being 0.56 mm and 0.45 mm, respectively. These unparalleled accurate results highlight that deep learning integrated into dental implantology workflow could significantly reduce manual labour in mandibular canal annotations.Peer reviewe

Reproducibility analysis of automated deep learning based localisation of mandibular canals on a temporal CBCT dataset

Funding Information: Expert Likert rating and error type reporting used in this study were partly provided by Antti Lehtinen, DDS Specialists of Dentomaxillofacial Radiology and Maarit Jordan DDS Resident of Dentomaxillofacial Radiology. Publisher Copyright: © 2023, Springer Nature Limited.Preoperative radiological identification of mandibular canals is essential for maxillofacial surgery. This study demonstrates the reproducibility of a deep learning system (DLS) by evaluating its localisation performance on 165 heterogeneous cone beam computed tomography (CBCT) scans from 72 patients in comparison to an experienced radiologist’s annotations. We evaluated the performance of the DLS using the symmetric mean curve distance (SMCD), the average symmetric surface distance (ASSD), and the Dice similarity coefficient (DSC). The reproducibility of the SMCD was assessed using the within-subject coefficient of repeatability (RC). Three other experts rated the diagnostic validity twice using a 0–4 Likert scale. The reproducibility of the Likert scoring was assessed using the repeatability measure (RM). The RC of SMCD was 0.969 mm, the median (interquartile range) SMCD and ASSD were 0.643 (0.186) mm and 0.351 (0.135) mm, respectively, and the mean (standard deviation) DSC was 0.548 (0.138). The DLS performance was most affected by postoperative changes. The RM of the Likert scoring was 0.923 for the radiologist and 0.877 for the DLS. The mean (standard deviation) Likert score was 3.94 (0.27) for the radiologist and 3.84 (0.65) for the DLS. The DLS demonstrated proficient qualitative and quantitative reproducibility, temporal generalisability, and clinical validity.Peer reviewe

Evaluation of virtual handles for dental implant manipulation in virtual reality implant planning procedure

Purpose: Many surgical complications can be prevented by careful operation planning and preoperative evaluation of the anatomical features. Virtual dental implant planning in three-dimensional stereoscopic virtual reality environment has advantages over three-dimensional projections on two-dimensional screens. In the virtual environment, the anatomical areas of the body can be assessed and interacted with in six degrees-of-freedom. Our aim was to make a preliminary evaluation of how professional users perceive the use of the virtual environment on their field. Methods: We prepared a novel implementation of a virtual dental implant planning system and conducted a small-scale user study with four dentomaxillofacial radiologists to evaluate the usability of direct and indirect interaction in a planning task. Results: We found that all four participants ranked direct interaction, planning the implant placement without handles, to be better than the indirect condition where the implant model had handles. Conclusion: The radiologists valued the three-dimensional environment for three-dimensional object manipulation even if usability issues of the handles affected the feel of use and the evaluation results. Direct interaction was seen as easy, accurate, and natural.publishedVersionPeer reviewe

Evaluation of voice commands for mode change in virtual reality implant planning procedure

Purpose: In dental implantology, the optimal placement of dental implants is important to meet functional and aesthetic requirements. Planning dental implants in virtual three-dimensional (3D) environment is possible using virtual reality (VR) technologies. The three-dimensional stereoscopic virtual reality environment offers advantages over three-dimensional projection on a two-dimensional display. The use of voice commands in virtual reality environment to replace button presses and other simple actions frees the user’s hands and eyes for other tasks. Methods: Six dentomaxillofacial radiologists experimented using a prototype version of a three-dimensional virtual reality implant planning tool, and used two different tool selection methods, using either only button presses or also voice commands. We collected objective measurements of the results and subjective data of the participant experience to compare the two conditions. Results: The tool was approved by the experts and they were able to do the multiple-implant planning satisfactorily. The radiologists liked the possibility to use the voice commands. Most of the radiologists were willing to use the tool as part of their daily work routines. Conclusion: The voice commands were useful, natural, and accurate for mode change, and they could be expanded to other tasks. Button presses and the voice commands should be both available and used in parallel. The input methods can be further improved based on the expert comments.publishedVersionPeer reviewe

Comparison of Deep Learning Segmentation and Multigrader-annotated Mandibular Canals of Multicenter CBCT scans

Deep learning approach has been demonstrated to automatically segment the bilateral mandibular canals from CBCT scans, yet systematic studies of its clinical and technical validation are scarce. To validate the mandibular canal localization accuracy of a deep learning system (DLS) we trained it with 982 CBCT scans and evaluated using 150 scans of five scanners from clinical workflow patients of European and Southeast Asian Institutes, annotated by four radiologists. The interobserver variability was compared to the variability between the DLS and the radiologists. In addition, the generalization of DLS to CBCT scans from scanners not used in the training data was examined to evaluate the out-of-distribution generalization capability. The DLS had lower variability to the radiologists than the interobserver variability between them and it was able to generalize to three new devices. For the radiologists' consensus segmentation, used as gold standard, the DLS had a symmetric mean curve distance of 0.39 mm compared to those of the individual radiologists with 0.62 mm, 0.55 mm, 0.47 mm, and 0.42 mm. The DLS showed comparable or slightly better performance in the segmentation of the mandibular canal with the radiologists and generalization capability to new scanners

Comparison of deep learning segmentation and multigrader-annotated mandibular canals of multicenter CBCT scans

Funding Information: The project was partly supported by Business Finland under project “Digital and Physical Immersion in Radiology and Surgery”. Mandibular canal annotations used in this study were partly provided by Antti Lehtinen, D.D.S. and Mika Mattila, D.D.S., both Specialists of Dentomaxillofacial Radiology.Deep learning approach has been demonstrated to automatically segment the bilateral mandibular canals from CBCT scans, yet systematic studies of its clinical and technical validation are scarce. To validate the mandibular canal localization accuracy of a deep learning system (DLS) we trained it with 982 CBCT scans and evaluated using 150 scans of five scanners from clinical workflow patients of European and Southeast Asian Institutes, annotated by four radiologists. The interobserver variability was compared to the variability between the DLS and the radiologists. In addition, the generalisation of DLS to CBCT scans from scanners not used in the training data was examined to evaluate its out-of-distribution performance. The DLS had a statistically significant difference (p < 0.001) with lower variability to the radiologists with 0.74 mm than the interobserver variability of 0.77 mm and generalised to new devices with 0.63 mm, 0.67 mm and 0.87 mm (p < 0.001). For the radiologists’ consensus segmentation, used as a gold standard, the DLS showed a symmetric mean curve distance of 0.39 mm, which was statistically significantly different (p < 0.001) compared to those of the individual radiologists with values of 0.62 mm, 0.55 mm, 0.47 mm, and 0.42 mm. These results show promise towards integration of DLS into clinical workflow to reduce time-consuming and labour-intensive manual tasks in implantology.Peer reviewe

Association between Oral Pathology, Carotid Stenosis, and Oral Bacterial DNA in Cerebral Thrombi of Patients with Stroke

Background and purpose. Risk of acute ischemic stroke has been associated with carotid artery atherosclerosis as well as with periodontal disease. We studied whether oral pathology or carotid atherosclerosis was associated with the presence and quantity of bacterial DNA in their aspirated thrombi. Methods. Thrombus aspirates and control arterial blood were taken from 71 patients (70.4% male; mean age, 67.4 years) with acute ischemic stroke. Tooth pathology was registered using CT scans. Carotid stenosis was estimated with CTA and ultrasonography. The presence of bacterial DNA from aspirated thrombi was determined using quantitative PCR. We also analyzed the presence of these bacterial DNAs in carotid endarterectomies from patients with peripheral arterial disease. Results. Bacterial DNA was found in 59 (83.1%) of the thrombus aspirates (median, 8.6-fold). Oral streptococcal DNA was found in 56 (78.9%) of the thrombus aspirates (median, 5.1-fold). DNA from A. actinomycetemcomitans and P. gingivalis was not found. Most patients suffered from poor oral health and had in median 19.0 teeth left. Paradoxically, patients with better oral health had more oral streptococcal DNA in their thrombus than the group with the worst pathology (p=0.028). There was a trend (OR 7.122; p=0.083) in the association of ≥50% carotid artery stenosis with more severe dental pathology. Oral streptococcal DNA was detected in 2/6 of carotid endarterectomies. Conclusions. Stroke patients had poor oral health which tended to associate with their carotid artery stenosis. Although oral streptococcal DNA was found in thrombus aspirates and carotid endarterectomy samples, the amount of oral streptococcal DNA in thrombus aspirates was the lowest among those with the most severe oral pathology. These results suggest that the association between poor oral health and acute ischemic stroke is linked to carotid artery atherosclerosis.publishedVersionPeer reviewe